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Abstract 

Recognising new or unusual features of an environ- 
ment is an ability which is potentially very useful 
to a robot. This paper demonstrates an algorithm 
which achieves this task by learning an internal rep- 
resentation of 'normality' from sonar scans taken 
as a robot explores the environment. This model 
of the environment is used to evaluate the novelty 
of each sonar scan presented to it with relation to 
the model. Stimuli which have not been seen before, 
and therefore have more novelty, are highlighted by 
the filter. The filter has the ability to forget about 
features which have been learned, so that stimuli 
which are seen only rarely recover their response 
over time. A number of robot experiments are pre- 
sented which demonstrate the operation of the filter. 

Keywords: Novelty Detection, Habituation, Mobile 
Robot, Self-Organisation 

1 Introduction 

Novelty detection, recognising when a particular 
stimulus has not been seen before, is a very use- 
ful ability for both animals and robots. This pa- 
per presents an algorithm which allows a robot to 
detect novel stimuli. The novelty filter described 
learns a representation of an environment and then 
detects deviations from that model by evaluating 
the novelty of each feature presented. 

A novelty filter has many potential uses on a mo- 
bile robot. For instance, it could be used as an 
attentional mechanism, directing the robot's atten- 
tion to newer features, which may be important 
and have not previously been learned [ pT[ . This 
reduces the amount of processing needed to deal 
with the robot's sensory perceptions. The novelty 
filter could also enable the robot to be used as an 
inspection agent. A model is built by the robot 
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of a 'clean' area, which has been inspected by hu- 
mans and is known to exhibit no undesirable fea- 
tures. The robot then explores the wider environ- 
ment and marks those stimuli which are not present 
in the model and therefore were not in the original 
environment. 

This paper demonstrates the behaviour of the 
novelty filter when the inputs to it are sonar scans 
taken while a robot explores an environment using 
a wall-following behaviour. One property of the 
novelty filter which is investigated here is the abil- 
ity to forget. This means that it will still find to be 
novel any stimuli which are seen only infrequently. 
This is useful because it ensures that these features 
are always considered novel, not learned over time. 
This can help the robot to deal with dynamic en- 
vironments, where things may change over time. If 
an event happens only occasionally we would like it 
to be considered novel, but without forgetting the 
robot will learn to recognise it no matter what the 
time interval between occurrences. 

1.1 Related Work 

The best known example of a novelty detector is 
the Kohonen Novelty Filter ||, |). This is an au- 
toencoder neural network which is trained using 
backpropagation of error Q . Once the network has 
been trained, presenting an input to the network 
produces one of the learned outputs, and taking 
the bitwise difference between the two displays the 
novel components of the input. 

A number of other researchers have proposed 
novelty filters. Ypma and Duin [^2) proposed a 
novelty detector based on the self-organising map. 
Training data was used to train the map, so that 
the data formed organised neighbourhoods. Then, 
when any data caused a neuron to fire which was 
beyond a predefined threshold from any of the 
neighbourhoods, the data was considered to be 
novel. This technique depends very strongly on the 
choice of threshold and assumes that the data pre- 
sented to the network formed strictly segmented 



neighbourhood clusters. The technique has been 
used by Taylor and Maclntyre |1| to detect ma- 
chinery faults. The network was trained on data 
recorded while the machine was operating without 
problems, and data deviating from this pattern was 
taken as novel. 

The technique of training the network on 'nor- 
mal' data and then attempting to recognise 
whether inputs come from the learned probability 
distribution is a common one when there is little 
data from a particular class, such as machine faults, 
but lots of data from the other classes. It has been 
used for topics as diverse as mammogram scans Q 
to machine breakdowns |l2|, |2lJ . 

An alternative method was proposed by Ho and 
Rouat H whose model is based on an integrate- 
and-fire network. The algorithm times how long it 
takes the oscillatory network to settle to a stable so- 
lution, reasoning that inputs which have been seen 
previously will converge faster than novel ones. 

2 The Novelty Filter 
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Figure 1: Left: An example of how the synaptic efficacy 
drops when habituation occurs. In both curves, a constant 
stimulus S(t) = 1 is presented, causing the efficacy to fall. 
The stimulus is reduced to S(t) — at time t = 60 where 
the graphs rise again, and becomes S(t) = 1 again at 
t = 100, causing another drop. The two curves show the 
effects of varying r in equation |^. It can be seen that a 
larger value of r causes both the learning and forgetting 
to occur faster. The other variables were the same for 
both curves, a — 1.05 and yo = 1.0. 



The novelty filter described in this paper works on 
the principle that something is novel if it has not 
been seen before. The question is how to recog- 
nise that an item is new. If we know in advance 
what everything in the environment will look like 
then it is relatively simple to train the robot to 
recognise each of those features. However, this is 
usually not possible. Instead, if the robot learns 
to ignore anything which it has seen before, then, 
it will only respond to novel things. This is some- 
thing which animals do quite well ||l^] . There are 
then two parts to the desired system - learning to 
recognise features that have been seen before, and 
evaluating their novelty. The first part, recognis- 
ing features, is a pattern recognition problem and 
has been considered widely in the neural network 
literature. One possible solution is the Kohonen 
Self- Organising Map ||, which is described below. 
The second problem, how to evaluate the novelty, 
is considered in section |2.2| . 

2.1 The Self-Organising Map 

The Self-Organising Map (SOM) of Kohonen f§ is 
a clustering mechanism which clusters input vec- 
tor in a topological way, so that perceptions which 
are similar excite similar regions of the network. 
The SOM is used here to perform Learning Vector 
Quantisation, choosing a winning neuron that best 
matches the input and moving that neuron and its 
neighbours closer to the input vector. It does this 
by selecting the neuron with the minimum distance 



between itself and the input. The distance is de- 
fined by: 

N-l 

d=£( Wi (t)-v(i)) 2 , (1) 

i=0 

where v(t) is the input vector at time t, w ; the 
weight between input i and the neuron and the sum 
is over the N components of the input vector. The 
weights for the winning neuron and its eight topo- 
logical neighbours are updated by: 

Wi(f + 1) = w,(() + r)(t) (v(t) - w,(t)) (2) 

where r\ is the learning rate, < rj{t) < 1. A square 
map field, comprising 100 neurons arranged in a 10 
by 10 grid, was used in the experiments reported 
here. The neighbourhood size was kept constant at 
±1 unit and the learning rate rj was 0.25, so that 
the network was always learning. 

2.2 Evaluating the Novelty 

Once a feature has been classified using the SOM, 
the novelty filter needs to assign a novelty value 
to the reading. A simple counter could be kept on 
each neuron, recording the number of times that 
each neuron has fired, and the output reduced ac- 
cordingly. This is biologically implausible and does 
not allow for any forgetting of stimuli. When an an- 
imal stops responding to a feature which has been 
presented to it repeatedly, the animal is said to have 
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Figure 2: The novelty filter. The input layer connects 
to a clustering layer which represents the feature space, the 
winning neuron (i.e., the one 'closest' to the input) passing 
its output along a habituable synapse to the output neuron 
so that the output received from a neuron reduces with 
the number of times it fires. 

habituated to the signal. Habituation, thought to 
be one of the simplest forms of plasticity in the 
brain , has been detected in a wide range of ani- 
mals from the sea slug Aplysia jl], U to humans ju] . 
The increase in the response to an habituated stim- 
ulus when the stimulus is withdrawn is called disha- 
bituation. It is thought to be a separate process 
acting on the habituable synapses 0). 

Several researchers have proposed models of the 
phenomenon of habituation, including Groves j5j, 
Wang and Hsu [^0) and Stanley |l6| . It is the model 
of Stanley, described below, which is used here. The 
synaptic efficacy, y(t), decreases according to the 
following equation: 

=a[y - y(t)} - S(t), (3) 

where j/o is the original value of y, r and a are time 
constants governing the rate of habituation and re- 
covery respectively, and S is the stimulus presented. 
The activity of the winning neuron and its neigh- 
bours are propagated up the synapse, so the input 
is S(t) = d (d defined in equation [l]) . Using equa- 
tion H| we can control how strongly a synapse re- 
sponds to an input. The first time a synapse fires 
its value is high, but each time it is used its strength 
decreases, as can be seen in the graph in figure [l]. 
Neurons which do not belong to the winning neigh- 
bourhood give an input of S(t) — to the synapse. 
This has the affect of causing the efficacy of the 
synapse to increase, or 'forget' some of its inhibi- 
tion, dishabituation. 
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Figure 3: The Nomad 200 mobile robot. 

2.3 Putting it all together 

By attaching an habituable synapse to each of the 
neurons in the SOM, a novelty filter is produced. 
The network is shown in figure ^. The only remain- 
ing question is how the constants a and r should be 
chosen. In order for the network to learn quickly, 
the synapse of the winning neuron should habit- 
uate rapidly. By choosing a value of t = 3.33, 
the synapse decreases to below 90% of its original 
value within 5 iterations. The neighbourhood neu- 
rons, which recognise similar perceptions, have a 
smaller amount of habituation, r = 14.33 and the 
other neurons, which are forgetting, have a longer 
time period, r = 100. This is because we do not 
want the network to forget perceptions too rapidly. 
Using this value a perception will recover from com- 
plete habituation in about 280 presentations. 

3 Experiments 

The experiments presented investigate the ability 
of the novelty filter to learn a model of an exter- 
nal environment through periodic sonar scans taken 
while exploring, and to detect deviations from that 
model. The effects of the forgetting mechanism are 
demonstrated. 

3.1 The Robot 

A Nomad 200 mobile robot (shown in figure |||) 
was used to perform the experiments. The band of 
infra-red sensors mounted at the bottom of the tur- 
ret of the robot were used to perform a pre-trained 
wall-following routine and the 16 sonar sen- 
sors at the top of the turret were used to provide 
perceptions of the robot's environment. The an- 
gle between the turret and base of the robot was 
kept fixed. The input vector to the novelty filter 
consisted of the 16 sonar sensors, each normalised 
to be between and 1, were thresholded at about 

4 metres. The readings were inverted so that inputs 
from sonar responses received from closer objects 
were greater. 



3.2 Experimental Procedure 

The experiments each consisted of a number of tri- 
als. In each trial, the robot started from an ar- 
bitrarily chosen starting point, and moved using a 
wall-following behaviour. Every 10 cm along the 
route, the smoothed readings from the sonar sen- 
sors were presented to the novelty filter, which pro- 
duced a novelty reading. Once the robot had trav- 
elled 10 m it stopped and saved the neural network 
weights. The robot was then returned to the start- 
ing point using manual control and the same pro- 
cedure repeated with the updated network weights. 

After each training trial, where the novelty filter 
learned about the environment, the learning mech- 
anism was turned off and a non-learning trial per- 
formed. The sonar inputs still generated output 
from the novelty filter to record the novelty of per- 
ceptions, but the robot did not learn. 

3.3 Environments 

Two environments were used in the experiments, 
together with a control environment for training. 
The two environments are shown in figure |4| They 
are similar sections of corridor on the second floor 
of the Computer Science building at the University 
of Manchester. The corridors are 1.7 m wide and 
have walls made from painted breezeblock. Doors 
made of varnished wood lead from the corridors 
into offices. 

4 Results 

4.1 Experiment One 

The first experiment aimed to demonstrate the abil- 
ity of the novelty filter to learn a representation 
of an environment and recognise novel features, so 
that the robot could be used as an inspection agent. 
The novelty filter was initialised randomly and then 
the robot was put into environment A. The left of 
figure |^ shows the results of this. The figures show 
the response of the output neuron to the input vec- 
tor of sonar readings that it receives every 10 cm 
along the route it travelled. At the top of the fig- 
ure is a diagram of the environment that the robot 
was travelling in at the time. Initially it can be 
seen that everything is novel, but where the robot 
perceives only wall, it rapidly learns to recognise 
this. The next thing that it notices is the crack in 
the wall on its right and then the doors. It is inter- 
esting to note that the robot finds the first crack 
more novel in the third run than in the second. 
This is because perceptions of such small features 
vary greatly depending upon the precise position of 
the robot. Only the cracks and the doorways are 



highlighted in the later trials. After two more learn- 
ing trials, the novelty filter has learned an accurate 
representation of the environment, as can be seen 
from the lack of response from the output neuron 
in trial 6. 

Once the novelty filter stopped finding anything 
novel in environment A, the robot was moved into 
environment B. This is a similar environment to A 
(see figure [|). The right of figure [| shows the re- 
sults of this. The only things which the robot finds 
to be novel in this environment are the perceptions 
of the doorways. This is because the doors are inset 
further into the wall in this environment. The con- 
trol trial demonstrates the responses of the novelty 
filter when the robot is put into environment B af- 
ter training in a control environment. For this the 
robot was driven around in an open area, travelling 
close to a wall, into the open space and back to the 
wall. It can be seen that the robot finds the en- 
vironment to be considerably more novel after this 
training. 

4.2 Experiment Two 

The second pair of experiments (shown in figure |6|) 
were designed to show the behaviour of the forget- 
ting part of the novelty filter. Two different experi- 
ments were performed. In both the network weights 
learned when exploring environment A were used. 
The first trial shows that the novelty filter had 
learned about this environment, since the robot did 
not find anything novel. A door in the environment 
was then opened (shown as Environment A* at the 
bottom of figure ||), and the robot learned about 
this new environment. A cardboard box was placed 
in the doorway. This was of sufficient height to be 
seen by the infra-red sensors which were respon- 
sible for the wall-following, but not by the sonar 
sensors. After each learning trial in this environ- 
ment, the door was closed and a non- learning trial 
in environment A was performed. The figures show 
that while no other features are detected, the open 
door is initially novel, but is learned over the three 
trials, and the closed door is initially recognised but 
is found progressively more novel as the novelty fil- 
ter forgets about this perception (since the filter is 
not learning when the robot perceives the closed 
door). 

The right of figure ^ shows the second experi- 
ment. A similar technique was used, again starting 
with the network weights learned in environment 
A in the first experiment. The robot learned about 
environment B, and after each learning trial was 
returned to environment A for a non-learning trial. 
Similar results can be seen - the robot initially finds 
parts of environment B novel, but learns to recog- 
nise it over the trials, while environment A, which 
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Figure 4: Diagrams of the environments used. The robot is shown facing in the direction of travel adjacent to the 
wall that it followed. The environments are two similar sections of corridor. The photographs show the environments as 
they appear from the starting position of the robot. The notice boards which can be seen in environment B are above the 
height of the robot's sonar sensors, and were therefore not detected. 



is recognised at first, becomes more novel. Obvi- 
ously, only particularly features of environment A 
are found novel, those which are not also seen in en- 
vironment B. These are the crack in the wall near 
the beginning of the environment and the doorway, 
which is set into the wall less than those in envi- 
ronment B. 

5 Summary and Conclusions 

The experiments described have demonstrated that 
the novelty filter can be used to learn a model of an 
environment and detect deviations from this model. 
The second experiment demonstrated the ability 
of the filter to forget perceptions that have been 
learned previously. This means that the novelty fil- 
ter will find novel features which are seen only occa- 
sionally or not seen for a long time. Therefore it can 
be trained in dynamic environments, where unfore- 
seen and undesirable perceptions, such as people 
walking past the robot, can occur. 

There are a number of areas which need further 
investigation. The integration of a number of ad- 
ditional sensory systems will allow the filter to be 
more widely applicable. In particular, the output 
of a monochrome CCD camera will be used. The 
images will need to be extensively preprocessed be- 



fore being presented to the novelty filter to reduce 
computational time. In addition, an investigation 
into alternatives to the Self-Organising Map (SOM) 
used in this work is underway. There are a number 
of well documented problems with the SOM, such 
as the fact that the size of the network needs to be 
prc-dctcrmincd, which means that the network can 
fill up, so that novel stimuli are incorrectly recog- 
nised as familiar. One possible solution is to use a 
growing network such as the Growing Neural Gas 
of Fritzke |Jj, another is to use a Mixture of Ex- 
perts J?J , with each expert learning a representation 
of a particular feature and voting on the novelty of 
perceptions. A committee of networks, where net- 
works of varying sizes and training regimes vote on 
the response to a particular input |]l0[ could also 
be used. 



Acknowledgements 

This research is supported by a UK EPSRC Stu- 
dentship. 



References 



fD 

3 

< 
o 

3 

3 

fD 
3 
r+ 

CO 



3 3 hrt 
o o e . 

rD n> 



C3 

CD 



O 
O" 

o 



Environment A 



< 

Q_ 



fD 






i — 1 


r+ 




n 
o 


— 1 


3 


3" 


r+ 


fl> 


o_ 


— ti 




3 


fD 


QJ 






< 


XJ 




n" 


o 


r+ 


3 


c 




0) 


ent 


on 




r+ 




3" 












cm' 




3" 




r+ 




Ul 




3" 




o 








(/) 




r+ 




3" 












fD 




l/l 




C 




r-f 




(/) 




o 




-h 




5 




< 




fD 




(/) 




rr 




cm' 




QJ 




r+ 




3' 




cm 



^ I 

3 fD 
- * 

o 



fD 

fjq 



I 1 

3" 3 

fD - 

o w 

g 3 



3 O 
ft S 



"O 

O 



ft -a 
o £ 



"a 
H "° 

3T "> 
m a. 



ft rs 



G 



Robot Trajectory 



10 



Output of Output Neuron 

Trial 1 (Learning) 



Distance (m) 



k 

0. ) 



0.5 



0.5 



Trial 2 (Not Learning) 



Trial 3 (Learning) 



Trial 4 (Not Learning) 



Trial 5 (Learning) 



Trial 6 (Not Learning) 



Environment B 



Q 



Robot Trajectory 



Output of Output Neuron 



Trial 1 (Robot accustomed to env. A and not learning) 



10 



Distance (m) 



Trial 2 (Learning) 



Trial 3 (Not Learning) 



Trial 4 (Learning) 



Trial 5 (Not Learning) 



Control Trial (Robot trained in different environment) 



u?. 


3 


3/ 


Q 




0) 






QJ 


o 


3 


< 


Q_ 








QJ 




n> 


5" 


Q_ 


r+ 


(/)' 


=r 


CU 


(/>' 


(/) 




I/) 




fD 


QJ 


el- 


r+ 




c 


s' 








ec 


> 


rr 




on 


imil 




QJ 


ho 










x 












— ■ 








05 




3 




p+ 




<7T 




Ul 




3- 




O 




< 
< 








O 




3 




r+ 




3~ 




<T> 




3_ 




(T3 




=r 




r+ 




cr 




c 




r+ 




c 




(/>_ 




3' 




cm 








3 




< 








O 




3 








<D 




3 




r+ 




(/> 




> 




QJ 




3 




Q_ 




CD 




— 1 




3" 




(D 




-% 




(T> 




(/) 




C 




r-f 




(/) 




QJ 




-V 



> <3 

1 era' 

2 G2 



a 2 



ID Q- 
Q. _ 



o 
ffq 



Q- 3 

o <ro 
o ' 



O ID 

< ^ 
=3 op" 



ID % 



<< V 3- 



ID M 



O 
O 



o 
3 

n> 



O 
ft o 



O I" < 
fl) 3 O 

~1 <""l" <Tl 



Environment A 



O 



Robot Trajectory 



Ou tput of Output Neuron 

Trial 1 (Not Learning - Env A) 

Jj 1 1 Ill 1 

Trial 2 (Learning and Forgetting - Env A*) 



Trial 3 (Not Learning - Env A) 



Distance (m) 



Trial 4 (Learning and Forgetting - Env A*) 



Trial 5 (Not Learning - Env A) 



Trial 6 (Learning and Forgetting - Env A*) 



Environment A* 



Environment 13 



© 



Robot Trajectory 



Output of Output Neuron 



Trial 1 (Learning and Forgetting - Env B) 



Trial 2 (Not Learning- Env A) 



Trial 3 (Learning and Forgetting - Env B) 



Trial 4 (Not Learning - Env A) 



Trial 5 (Learning and Forgetting - Env B) 



Trial 6 (Not Learning - Env A) 



Environment A 



Distance (m> 



© 



[1] Craig Bailey and M.C. Chen. Morphological 
basis of long-term habituation and sensitiza- 
tion in aplysia. Science, 220:91-93, 1983. 

[2] CM. Bishop. Neural Networks for Pattern 
Recognition. Oxford University Press, Oxford, 
England, 1995. 

[3] Bernd Fritzke. A growing neural gas network 
learns topologies. In G. Tesauro, D. S. Touret- 
zky, and T. K. Leen, editors, Advances in Neu- 
ral Information Processing Systems 7, pages 
625-632, Cambridge, MA, 1995. MIT Press. 

[4] S. Greenberg, V. Castellucci, H. Bayley, and 
J. Schwartz. A molecular mechanism for long- 
term sensitisation in aplysia. Nature, 329:62- 
65, 1987. 

[5] P.M. Groves and R.F. Thompson. Habitua- 
tion: A dual-process theory. Psychological Re- 
view, 77(5):419-450, 1970. 

[6] Tuong Vinh Ho and Jean Rouat. Novelty de- 
tection based on relaxation time of a network 
of integrate-and-fire neurons. In Proceedings 
of the 2nd IEEE World Congress on Compu- 
tational Intelligence, WCCI'98, pages 1524- 
1529, 1998. 

[7] Michael I. Jordan and Robert A. Jacobs. Hier- 
archical mixtures of experts and the EM algo- 
rithm. Neural Computation, 6:181 - 214, 1994. 

[8] Teuvo Kohonen. Self-Organization and Asso- 
ciative Memory, 3rd ed. Springer, Berlin, 1993. 

[9] Teuvo Kohonen and E. Oja. Fast adaptive for- 
mation of orthogonalizing filters and associa- 
tive memory in recurrent networks of neuron- 
like elements. Biological Cybernetics, 25:85-95, 
1976. 

[10] Anders Krogh and Jesper Vedelsby. Neural 
network ensembles, cross validation, and ac- 
tive learning. In G. Tesauro, D.S. Touretzky, 
and T.K. Leen, editors, Advances in Neural 
Information Processing Systems 7, pages 231 
- 238. MIT Press, 1995. 

[11] Stephen Marsland, Ulrich Nchmzow, and 
Jonathan Shapiro. Novelty detection for robot 
neotaxis. In Proceedings of the 2nd Inter- 
national Symposium on Neural Compuatation 
(NC'2000), pages 554 - 559, 2000. 

[12] Alexandre Nairac, Neil Townsend, Roy Carr, 
Steve King, Peter Cowley, and Lionel 
Tarassenko. A system for the analysis of jet 
system vibration data. Integrated Computer- 
Aided Engineering, 6(1), 1999. 



[13] Ulrich Nehmzow. Autonomous acquisition of 
sensor-motor couplings in robots. Technical 
Report UMCS-94-11-1, Department of Com- 
puter Science, University of Manchester, 1994. 

[14] John O'Keefe and Lynn Nadel. The Hippocam- 
pus as a Cognitive Map. Oxford University 
Press, Oxford, England, 1977. 

[15] Michael P. Perrone and Leon N. Cooper. When 
networks disagree: Ensemble methods for hy- 
brid neural networks. In R.J. Mammone, ed- 
itor, Neural Networks for Speech and Image 
Processing. Chapman-Hall, 1993. 

[16] James C. Stanley. Computer simulation of a 
model of habituation. Nature, 261:146-148, 
1976. 

[17] L. Tarassenko, P. Hayton, N. Cerneaz, and 
M. Brady. Novelty detection for the identi- 
fication of masses in mammograms. Artificial 
Neural Networks, 1995. 

[18] Odin Taylor and John Maclntyrc. Adaptive lo- 
cal fusion systems for novelty detection and di- 
agnostics in condition monitoring. In SPIE In- 
ternational Symposium on Aerospace/ Defense 
Sensing, 1998. 

[19] R.F. Thompson and W.A. Spencer. Habitu- 
ation: A model phenomenon for the study of 
neuronal substrates of behaviour. Psychologi- 
cal Review, 73(l):16-43, 1966. 

[20] DeLiang Wang and Chochun Hsu. SLONN: 
A simulation language for modelling of neural 
networks. Simulation, 55:69-83, 1990. 

[21] K. Worden, S.G. Pierce, G. Manson, W.R. 
Philp, W.J. Staszewski, and B. Culshaw. De- 
tection of defects in composite plates using 
lamp waves and novelty detection. Interna- 
tional Journal of Systems Science, 1999. 

[22] Alexander Ypma and Robert P. Duin. Novelty 
detection using self-organizing maps. In Pro- 
ceedings of International Conference on Neural 
Information Processing and Intelligent Infor- 
mation Systems, 1997. 



